Deictic Believability: Coordinated Gesture, Locomotion, and Speech in Lifelike Pedagogical Agents

Lifelike animated agents for knowledge based learning environments can provide timely cus tomized advice to support students problem solving Because of their strong visual presence they hold signi cant promise for substantially increasing students enjoyment of their learning experiences A key problem posed by lifelike agents that inhabit arti cial worlds is deictic believability In the same manner that humans refer to objects in their environment through judicious combinations of speech locomotion and gesture animated agents should be able to move through their environment and point to and refer to objects appropriately as they provide problem solving advice In this paper we describe a framework for achieving deictic believabil ity in animated agents A deictic behavior planner exploits a world model and the evolving explanation plan as it selects and coordinates locomotive gestural and speech behaviors The resulting behaviors and utterances are believable and the references exhibit a lack of ambiguity This approach to spatial deixis has been implemented in a lifelike animated agent Cosmo who inhabits a learning environment for the domain of Internet packet routing Cosmo provides realtime advice to students as they escort packets through a virtual world of interconnected routers Results of an informal focus group study with the Cosmo agent suggest that the spatial deixis framework produces clear explanatory animated behaviors